Harnessing Disagreement for Event Semantics
نویسندگان
چکیده
The focus of this paper is on how events can be detected & extracted from natural language text, and how those are represented for use on the semantic web. We draw an inspiration from the similarity between crowdsourcing approaches for tagging and text annotation task for ground truth of events. Thus, we propose a novel approach that harnesses the disagreement between the human annotators by defining a framework to capture and analyze the nature of the disagreement. We expect two novel results from this approach. On the one hand, achieving a new way of measuring ground truth (performance), and on the other hand identifying a new set of semantic features for learning in event extraction.
منابع مشابه
CrowdTruth: Machine-Human Computation Framework for Harnessing Disagreement in Gathering Annotated Data
In this paper, we introduce the CrowdTruth open-source software framework for machine-human computation, that implements a novel approach to gathering human annotation data in a wide range of annotation tasks and on a variety of media (e.g. text, images, videos). The CrowdTruth approach captures human semantics through a pipeline of three processes: a) combining various machine processing of te...
متن کاملCrowdsourcing Ambiguity-Aware Ground Truth
The process of gathering ground truth data through human annotation is a major bottleneck in the use of information extraction methods. Crowdsourcing-based approaches are gaining popularity in the attempt to solve the issues related to volume of data and lack of annotators. Typically these practices use inter-annotator agreement as a measure of quality. However, this assumption often creates is...
متن کاملModeling and Evaluation of Stochastic Discrete-Event Systems with RayLang Formalism
In recent years, formal methods have been used as an important tool for performance evaluation and verification of a wide range of systems. In the view points of engineers and practitioners, however, there are still some major difficulties in using formal methods. In this paper, we introduce a new formal modeling language to fill the gaps between object-oriented programming languages (OOPLs) us...
متن کاملModeling and Evaluation of Stochastic Discrete-Event Systems with RayLang Formalism
In recent years, formal methods have been used as an important tool for performance evaluation and verification of a wide range of systems. In the view points of engineers and practitioners, however, there are still some major difficulties in using formal methods. In this paper, we introduce a new formal modeling language to fill the gaps between object-oriented programming languages (OOPLs) us...
متن کاملEmotional disagreement The role of semantic content in the expression of, and disagreement over, emotional values
The role of semantic content in the expression of, and disagreement over, emotional values Isidora Stojanovic, Institut Jean-Nicod – CNRS – ENS – EHESS Abstract When we describe an event as sad or happy, we attribute to it a certain emotional value. Attributions of emotional value depend essentially on an agent (and on his or her emotional responses); and yet, people readily disagree over such ...
متن کامل